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Abstract. Let G be an afRne algebraic group and let X be an affine algebraic 
variety. An action G x X ^ X is called observable if for any G-invariant, proper, 
closed subset Y oi X there is a nonzero invariant / G lk[X]'^ such that /|^ =0. 
We characterize this condition geometrically as follows. The action G x X ^ X is 
observable if and only if (1) there is a nonempty open subset U Q X consisting of 
closed orbits, and (2) the field k(X)'' of G-invariant rational functions on X is equal 
to the quotient field of k[X]'^. In case G is reductive, we conclude that there exists 
a unique, maximal, G-stable, closed subset Xsoc of X such that G x Xgoc Xsoc 
is observable. Furthermore, the canonical map Xsod/G X//G is bijective. 



1. Introduction 

A closed subgroup H of the affine algebraic group G is called an observable sub- 
group if the homogeneous space G/H is a quasi- affine variety. Such subgroups have 
been introduced by Bialynicki-Birula, Hochschild and Mostow in [2j, and researched 
extensively since then, notably by F. Grosshans. See ^12j for a survey on this topic, 
and Theorem 12.31 below for other useful characterizations of observable subgroups. 

In this paper we develop further this point of view, and define the notion of an 
observable action of G on the affine variety X. Our main results pertain to actions 
of non-reductive groups. But we also investigate the situation where G is reductive. 

To state our results we ffist introduce some notation. Let k be an algebraically 
closed field. We work with affine algebraic varieties X over k. An algebraic group is 
assumed to be a smooth, affine, group scheme of finite type over k. If X is an affine 
variety over k we denote by k[X] the ring of regular functions on X. If / C k[X] 
is an ideal, we denote by V(/) = {x e X : f{x) = OV/g/}. IfFcAisa 
subset, we denote by I{Z) ={f e k[X] : f{y) = OWyeY}. If f e k[X], we denote 
by Xf = |a; G A : f{x) ^ O}. Morphisms Lp : X Y between affine varieties 
correspond to morphisms of algebras k[Y] k[A], hy if ^ if* , <f*{f) = f o ip. If 
X is irreducible we denote by k(A) the field of rational functions on X. If A is any 
integral domain we denote by [A] its quotient field. Thus if X is an affine variety, 
then k(A) = [k[A]] . If G x A ^ A is a regular action of G on A we consider the 
induced action of G on k[A], defined as follows. For / G k[A] and g E G we set 
{g ■ f){x) = f\g^^x). It is well known that G-stable closed subset of A correspond to 
G-stable radical ideals of k[A]. If x G A we will denote by 0{x) = {g ■ x : g E G} the 
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G-orbit of X. We say that / G k[X] is G-invariant ii g ■ f = f for any g & G. The set 
of G-invariants k[X]'^ forms a k-subalgebra of Ik:[X], possibly non-finitely generated. 

In this paper we are interested in situations whereby an affine algebraic group acts 
on an irreducible, affine variety X in such a way that [k[X]'^] = k;(X)'^ = . 
When this happens, one can separate the orbits of maximal dimension generically 
with invariant regular functions. See for example [IHl Prop. 3.4]. In Theorem 13 . 71 and 
Corollary 13.91 we obtain a generalization of Igusa's criterion (see [13] and [IH §4.5]). 
We express our results in terms of the affinized quotient vr : X — >• X/ G, induced by 
the inclusion k[X]'^ C k[X]. This quotient, even when it does not coincide with the 
categorical quotient, contains enough information about the action to give us control 
of an open subset of orbits of maximal dimension. See Definition 12. 8[ 

We define a regular action G x X — > X, to be observable if for any nonzero G- 
stable ideal / C k[X] we have I'^ 7^ (0). A closed subgroup if of G is an observable 
subgroup, in the sense of Grosshans above, if and only if the action H x G ^ G, hy 
left translations, is an observable action. It turns out that, in this situation, the left 
action is observable if and only if the right action is so. 

If G X X —> X is an observable action of G on the affine variety X, then it follows 
easily that X contains a dense open subset consisting entirely of closed G-orbits. 
This is a sufficient condition for the action to be observable if G is reductive, but not 
in general. In Theorem 13.101 the main result of this paper, we provide the following 
geometric characterization. The action G x X ^ X is observable if and only if (1) 
[k[X]^] = [k[X]] and (2) X contains a dense open subset consisting entirely of 
closed G-orbits. 

It seems natural then to consider the affinized quotient of an observable action. 
In this case, we show that there exists a nonzero invariant / G k[X]'^ such that the 
affinized quotient vr : X/ -^//afi C is a geometric quotient; see Theorem 13.181 

As an application, we consider the case of a reductive group G acting on the 
irreducible, affine variety X. In this case we prove that there exists a unique, maximal, 
G-stable, closed subset Xgoc C X such that the restricted action G x Xgoc — > Xsoc is 
observable. Furthermore, k[X]'^ C klXsoc]*^ is a finite, purely inseparable extension. 
In particular, the morphism Xsoc//G X//G is finite and bijective. See theorems 
gHandim 
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2. Preliminaries 

Let G be an affine algebraic group and X an algebraic variety. As usual, a (regular) 
action of G on X is a morphism : G x X X, denoted by ip{g, x) = g ■ x, such 
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that (ab) ■ x = a - {h- x) and 1 • a; = a; for all a, 6 G G and x G X. Since all the actions 
we will work with are regular, we will drop the adjective regular. 

Definition 2.1. Let G be an affine algebraic group and H G G a, closed subgroup. 
The subgroup H is observable in G if and only if for any nonzero if-stable ideal 
/ C k[G] we obtain that ^ (0). 

Remark 2.2. The notion of an observable subgroup was introduced by Bialynicki- 
Birula, Hochschild and Mostow in [2J. The characterization of observable subgroups 
by the condition "/^ ^ (0)" in Definition O appears for the first time in [TUl 
Definition 10.2.1]. 

Theorem 2.3. Let G be an affine algebraic group and H G G a closed subgroup. 
Then the following conditions are equivalent: 

(1) The subgroup H is observable in G. 

(2) The homogeneous space G/H is a quasi-affine variety. 

(3) For an arbitrary proper and closed subset C C G/H, there exists an nonzero 
invariant regular function f G klG]"^ such that f{C) = 0. 

Moreover, if G is connected the above conditions are equivalent to the condition 

(4) [k[G]]^= [k[G]^]. 

Proof. See for example [TUl Thm. 10.5.5]. □ 

Definition 2.4. Let G be an affine algebraic group acting on an affine variety X. 
We say that the action is observable, if for any nonzero G-stable ideal / C k[X], 
^ (0). 

Example 2.5. (1) Let G be an algebraic group and H a closed subgroup. Then the 
action of if on G by translations is observable if and only if H is observable in G. 

(2) Any action of the unipotent group U on an affine variety is observable, since for 
any nonzero fZ-module M, we obtain that ^ {0}. We will prove in Proposition 
12.71 below that this property characterizes unipotent groups. 

First, we recall the construction of the induced space: 

Definition 2.6. Let G be an algebraic group, and let a closed subgroup H d G act 
on an algebraic variety X. The induced space G *h X is defined as the geometric 
quotient of G x X under the ii-action h ■ {g, x) = {gh^^, h ■ x). 

Under mild conditions on X (e.g. X normal and covered by quasi-projective in- 
stable open subsets), this quotient exists. Clearly, G *h X is a. G- variety, for the 
action induced by a ■ {g, x) = {ag, x). We will denote the class of {g, x) in G x X by 
[g,x] G G X. We refer the reader to [1], where the notion of induced space was 
introduced by Bialynicki-Birula, and to [TH] , for a concise survey on this construction 
and its many properties, and resume in the next paragraphs the properties of the 
induced space we need in what follows. 

It is easy to see that G*hX is a G- variety, for the action a-[g, x] = [ag, x], a, g E G, 
xeX. 
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The projection G x X ^ G induces a morphism p : G *h X ^ G/H, [g, x] [g]. 
The morphism p is a G-equivariant fibration, with fiber 

X = p~\[i]) = {[i,x] ■.xex}cG*HX. 

Moreover, the G-stable closed subsets of G *h X correspond bijectively to if-stable 
closed subset of X, via the maps 

G*H X D Z Z nX and X dY ^GY cG*hX. 

Proposition 2.7. Let G be a connected ajfine algebraic group such that every action 
of G on an ajfine algebraic variety is observable. Then G is a unipotent group. 

Proof. We first observe that every G-orbit on an affine G- variety X is closed. Indeed, 
if (9 C X is an orbit, then the action of G on the affine variety O is observable, and 
hence O is closed. 

The result now follows from the fact that if an algebraic group G acts on any 
affine algebraic variety with closed orbits, then G is unipotent (see [9]). We give 
a short proof of this fact, for the sake of completeness. Assume that G contains a 
multiplicative subgroup k*, and consider the induced space G*k* k, where k* acts on 
k by multiplication. Then 1]) = {^[g,a] : g ^ G , a 7^ O} is an open orbit, and 

this is a contradiction. It follows that G is a unipotent algebraic group. □ 

Let G be an affine algebraic group acting on the affine variety X. Then it is well 
known that the categorical quotient does not necessarily exists, even when k[X]'-^ is 
finitely generated. However, if k[X]'^ is finitely generated then Spec(k[X]*^) satisfies 
a universal property in the category of affine algebraic varieties. 

Definition 2.8. Let G be an affine algebraic group acting on an affine variety X, in 
such a way that k[X]*^ is finitely generated. The affinized quotient of the action is 
the morphism vr : X ^ X/^^ G = Spec(k[X]^). 

It is clear that vr satisfies the following universal property: 

Let Z be an affine variety and f : X Z a morphism constant on the G-orbits. 
Then there exists an unique f : Xj G ^ Z such that the following diagram is 
commutative. 

X^ — 




Indeed, it is clear that the induced morphism /* : k[Z] k[X] factors through 
k[Xf. 

Notice that the affinized quotient is the affinization of the categorical quotient 
when the latter exists, and k[X]*^ is finitely generated. Indeed, if X — > X//G is the 
affine quotient then X/^^ G = Spec(C(X//G)). 

Remark 2.9. It is clear that the morphism vr : X — > X/^^ G is dominant. However, 
7T is not necessarily surjective. Consider a semisimple group G and its maximal 
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unipotent subgroup U. Then G/U = -k{G/U) is known to be a proper open subset 
of GU, U. 

Let G be an affine group acting on an affine variety X. It is very easy to prove 
that if / e k[X]^, then {k[Xf)^ = (k[X]/)^. This technical remark will allow 
us, combined with Propositions 13.21 and 13.31 below, to study affinized quotients of 
observable actions. 

Lemma 2.10. Let G be an affine algebraic group and X an affine G-variety such 
that k[X]'-^ is finitely generated. Let ir : X ^ ^/afi ^ affinized quotient. Then 

for any f G k[X]'-^, n\xf '■ Xj {Xj G)f is the affinized quotient. 

Proof. Indeed, {k[Xf) = {k[X]f)^ . □ 



3. Observable actions and affinized quotients 

In this section we study observable actions. In Theorem l3.10l we provide a geometric 
characterization of the observability of an action. This characterization allows us to 
construct an open subset of X with affine geometric quotient, see Theorem 13.181 

We begin by given some equivalent condition for an action to be observable. 

Lemma 3.1. Let G be a connected affine algebraic group acting on an affine variety 
X, and let X = [J'^^^Xi be the decomposition of X in irreducible components. Then 
the action G x X ^ X is observable if and only if the restricted actions G x Xi Xi 
are observable for all i = 1, . . . ,n. 

Proof. Assume that the action of G on X is observable. We prove without loss of 
generality that G x Xi — > Xi is an observable action. Let F C Xi be a G-stable 
closed subset. Then Z = Y U |jr=2^« ^ G-stable closed subset, Z C X. Since the 
action G x X ^ X is observable, there exists / G \ {0}) C k[X]. It follows 

that /Ixi 7^ 0, and hence / e (X(F)'^ \ {0}) C k[Xi]. Thus, the restricted action 
G X Xi ^ Xi is observable. 

For the converse, consider the morphism vr : |J- Xj — X. Then vr induces an 
injective morphism A = k[X] — > nik[Xj] = UiBi = B. Let / C k[X] be a nonzero 
G-ideal. Consider /q = {/ G / | fB C A}. One checks easily that /q is a G-ideal 
of B. Now there is a regular element r & A such that A^. = B^ = nj(Sj)r-, where 
Ti = r\xi- One checks that 

{h)r = {/ e I fB, C A,} = Ir 

since A, = B,. Thus there exists m > such that r™/ C Jg. In particular, Iq ^ (0). 
But Jo = 0i/i, where C k[Xi] is a G-ideal. Hence = 0./f (0). But then 
^ (0). □ 

Proposition 3.2. Let G be a connected affine algebraic group acting on an affine 
variety X. Then the following are equivalent 

(1) The action is observable. 
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(2) For any nonempty G-stable open subs etU C X, there exists f e k[Xf \ {0} 
such that Xf C U. 

(3) For every f G k[X]'^, the action of G on Xf is observable. 

(4) There exists f G IsfX]*^ \ {0} such that f is not a zero divisor and that the 
action of G on Xf is observable. 

(5) For every G-stable nonzero prime ideal V o/k[X], V'~' ^ (0). 

(6) For every nonempty G-stable affine open subset U , the restriction G x U —>■ U 
is an observable action. 

Proof. We prove first the equivalence of conditions (1) - (6) in tlie case wlien X is 
an irreducible variety. 

The implication (1) <^==^ (2) is trivial. 

To prove that (1) =^ (3), let / G k[X]'^, and let / C be a nonzero G-stable 

ideal. Then J = I f] k[X] C k[X] is a nonzero G-stable ideal. If we let h e J'^ \ {0} 
then h = h/1 G J'^. 

The implication (3) =^ (4) is trivial. 

In order to prove that (4) =^ (5), let V C k[X] be a nonzero prime G-stable ideal. 
If / G P there is nothing to prove. If / ^ P, let Vf = Vk[X]f be the prime ideal 
generated by V. Since / G k[Xf, it follows that Vf is G-stable; let /i//" G Vf\{0}. 

Then 0^h = e {Vf n k[X])^ = V^. 

To prove that (5) ^> (1), let / C k[X] be a nonzero G-stable ideal. Then V(/) C X 
is a G-stable variety. Since G is connected, every irreducible component of V(/) is 
G-stable, and hence r{I), the radical of /, is an intersection of G-stable prime ideals 
r(/) =V,n---nVs. Let /. G Pf \ {0}, 2 = 1, . . . , s. Then ^ / = [l /. e r{lf, 
and thus there exists n > such that 7^ /" G J*-^. 

In order to prove (1) (6), let [/ C X be a nonempty G-stable affine open 
subset. Then by (2) there exists / G k[X]*^ such that Xf C U. By (4) it follows that 
the restriction G x Xf —>■ Xf is observable. Since Uf = Xf it follows that assertion 
(4) holds for the action of G on U, i.e. that the action on U is observable. 

Finally, if (6) holds then the action of G on Xf is observable for every / G k[X]*^. 
In particular, assertion (3) holds. 

Assume now that X is reduced and let X = [J-Xi be the decomposition in irre- 
ducible components. 

The implications (1) <^=^ (2) and (3) =^ (4) are trivial. 

To prove that (1) =^ (3), just observe that Xf = IJj(^i)/|x ' apply Lemma 

Finally, the remaining implications are proved following the same reasoning as in 
the irreducible case. □ 



We recall now two useful results that we will need in what follows. 
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Proposition 3.3 (Grosshans). Let G be an affine algebraic group acting on an affine 
variety. Then there exists f E k[X]'^ \ {0} such that [k.[X]f)'^ = (k[X]'^)^ is finitely 
generated. 

Proof. See [Hi Thm. 1]. □ 

Theorem 3.4 (Rosenlicht [ISIIIZ])- Let G be an algebraic group and X an irreducible 
G-variety. Then there exists a G-stable open subset ^ Xq C X such that the action 
of G restricted to Xq has a geometric quotient. 

Proof. See for example [101 §13.5]. □ 

If G X X ^ X is an observable action then the field of invariant rational functions is 
generated by invariant regular functions (see Theorem 13.101 below) . In [15] those au- 
thors provide some useful sufficient conditions to guarantee that [k[X]]'~^ = [k[X]'^]. 

The following theorem may be regarded as a generalization of "Igusa's criterion" . 
See [13] and [ISl §4.5]. 

Theorem 3.5. Let G be an algebraic group acting on an affine algebraic variety X . 
Assume that either 

(a) G^ , the connected component ofl, is solvable, or 

(b) k[X] is factorial. 

Then every rational function f G [k[X]]'~^ is the quotient of two semiinvariant 
regular functions. In other words, there exist h,g E k[X] and x £ ^{G) such that 
f = f^, a ■ g = x{C')9 ^^'^^ a ■ h = for all a E G. 

Moreover, if G has no nontrivial character (that is, ifk[G]* = k*), then [k[X]]*^ = 

Proof See [15^ Thm. 3.3]. □ 

Definition 3.6. It is well known that the set of points whose orbit has maximal 
dimension is an open subset, we denote it by 

^max = {xeX : dimO(x) > dimO{y) Vy G X}. 

We let 

Q{X) = |x G X : dim(9(x) is maximal and 0{x) = 0{x)^. 

Theorem 3.7. Let G be an affine algebraic group acting on an affine variety X , such 
that k[X]'^ is finitely generated, and let n : X —>■ Xj G be the affinized quotient. 
Assume that 

(1) [k[xr] = [k[x]f. 

Then 



(2) There exists a nonempty open subset W C X/^^G such that n (x) = 0{y), 
with y G Xjnax; for all x G W. Moreover, W can be taken of the form (X/ ^G)f = 
Xj/^^G, wtthfek[Xf. 
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Conversely, if (2) holds then [k[X]^] C [k[X]] is a purely inseparable finite 
extension. In particular, z/chark = 0, then [k[X]'^] = [k[X]]*^. 

Proof. In order to prove the implication (1) =^ (2), let Xq C X be as in Rosenlicht's 
Theorem 13. 41 and p : Xq Xq/G the geometric quotient. Then we have the following 
commutative diagram 



(1) 



Xn^ 



p 

Xo/G 



X 



Since 

k(Xo/G) ^ k(Xo)^ = k(X)^ = [k[X]]^ = [k[X]^] = 

[k[X/^,G]] =k(X/^,G), 

it follows that the geometric quotient Xq/G and the affinized quotient X/^^G are 
birationally equivalent via (p. Hence, there exists an open subset V C Xq/G such 
that (p\y : V — i> X/^jj G is an open immersion. Replacing X^hy U = -K^^iV) we can 
assume that if is an open immersion, and hence p = 'k\xq- 

Let V C X/^j,. G be an open subset such that, for each w & V , the fiber 7r~^(w) 
has all irreducible components of maximal dimension (as follows from Chevalley's 
Theorem, see for example p21 Thm. 1.5.4]). Let m = dimX — dimX/^^ G be the di- 
mension of such a fiber. It follows from diagram ([1]) that m is the maximal dimension 
of the orbits of G on X. Moreover, if w e W = (Xq/G) nV C X/^^ G, then there ex- 
ists an unique irreducible component Z C n^^{w) such that Z = 0{x), with x G Xq. 
Let Y C 7c~^{w) another irreducible component; then Y C X\Xo. Let A G W he the 
set of point which fiber is reducible, A = E W : it^^{w) ^ t\^^{w) fl Xq}. Since 
A C 7r(X \ Xq), it follows that if A is dense in Xq, then X \ Xq dominates X/^^. G. 
Hence the maximal dimension of a fiber of t^\x\xq dim(X \ Xq) — dimX/^^ G < 

m. But for a G A, dim(7r|^^ 



dim X — dim X/^^ G 
tion. Hence, for all w & W \ A ^ 



we have that vr^^ 



(w) 



m, which is a contradic- 
0{x), with X e Xq. 



Let now / e k[X]'^ such that {X/^^ G)f C W\A. Then X/ satisfies assertion (2). 

Assume now that (2) holds, with W = (X/^^. G)/, / G k[X]^. Then 7r-^{W) = X/, 

[k[X]^] = [k[X]^] = [k[X/]^] and [k[X]]^ = [k[X]/]^ = [k[X/]]^. Thus, we can 

assume without loss of generality that X is such that n^^iy) = G ■ x, with x G X^ax, 
for all y G X/^^. G. Let Xq C X be as in Rosenlicht's Theorem and consider the 
commutative diagram 



X 



^o/G — X/^g G 
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Let I/, ^ G Xq/G be such that (/?(?/) = ip{z). Then p ^{y) and p ^{z) are contained 
in 7i~^(^(p{y)), and hence p~^{y) = p~^{z). It follows that ip is injective. Since ip is 
dominant, it follows that ip induces a finite, purely inseparable extension [k[X]'^] C 
[k[X]]^. □ 

Remark 3.8. (1) Observe that if G is reductive, then k[X]'-^ is finitely generated, 
and thus Theorem 13.71 applies. 

(2) Let G X X X an action satisfying condition (1) of Theorem 13. 7^ and O, O' C 
-^max n Xf, with / as in condition (2) of that theorem. Then there exists h G 
k[Xf/^^G] = k[X]^ such that /(vr(0)) = and /(vr(C')) = 1- Hence k[X]^, or 
equivalently the affinized quotient n : Xf Xj/^^ G, separates orbits in Xmax HX/. 

Corollary 3.9. Let G be an algebraic group acting on an affine algebraic variety, in 
such a way that [k[X]^] = [k[X]] . Then there exists f G k[X]^ such that k[X]^ 
separates the orbits in X^ax H Xf. 

Proof. By Proposition 13. 3[ there exists h G k[X]'-^ such that k[Xh] = k[X]h is finitely 
generated. Since 

[k[X^f] = [k[Xf] = [k[X,]f 

it follows from Theorem [32] that there exists g G k[X/j]'^ = k[X]^ such that k[(X/j)g]'^ 
k[X]^^ separates the orbits in Xmax H Xhg. □ 

We now characterize observability in terms of the underlying geometry. 

Theorem 3.10. Let G be an affine group acting on an affine variety X. Then the 
action is observable if and only if (1) [k[X]'^] = [k[X]]'^ and (2) VL{X) contains a 
nonempty open subset. 

Proof. Assume that the action is observable and let F = X \ Xmax and let / G 
X{Y)^ \ {0}. Then the nonempty affine open subset Xf C Xmax is G-stable and 
any point x G Xf is such that dim(9(x) is maximal. Let O C Xf be an orbit. 
Since every orbit is open in its closure, it follows that ii O ^ O then C fl y 7^ 0. 
Since / is constant on orbit closures, it follows that /|o = and hence we obtain a 
contradiction. It follows that 7^ Xf C VL{X). 

Clearly [k[X]^] C [k[X]]^. Let g G [k[X]]^, and consider the ideal / = {/ G 
k[X] : fg G k[X]}. Clearly / is G-invariant, and hence there exists / G k[X]'-^ such 
that fg G k[X]^. 

In order to prove the converse, let / G k[X]'-^ such that k[Xj]*^ is finitely generated 
(see Proposition 13.31) . Then, by Proposition 13.21 the action of G on X is observable 
if and only if the action on Xf is so. Thus, we can assume without loss of generality 
that k[X]'-^ is finitely generated. Let tt : X — X/ G be the affinized quotient. 
By Theorem 13. 7[ there exists / G k[X]*^ such that 7r^^(?/) = G ■ x for aX\ y & V = 
{X/ G)f = Xf/^g. G. Moreover, we have the following commutative diagram 
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(X^)o c ^ X;C . X 



p 



G 

where p is a geometric quotient. Since k((Xy)o/G) = k((Xj)o) = k(Xj)^, it follows 
by hypothesis that k((X/)o/G') = k(X//^g G). Since p and vr separate closed orbits, 
it follows that ip is an open immersion. 

Since Vt{X) contains a nonempty open subset, it follows that i7(X)n(X/)o 7^ 0. Let 
g e k[Xf such that (X^/^^ G), C (X^)o/G. If y G Xj/^^ G)g, then Tr^Hy) = 0(x), 
where x G f2(X) fl (Xj)o, hence n~^{y) is a closed orbit of maximal dimension. 
Therefore, nl^^ : Xj^ — * {Xf / G)g = X^gj G is such that all its fibers are closed 
orbits. Replacing X by X/^, we can hence assume that all the fibers of the afiinized 
quotient are closed orbits. In this situation, it is clear that any nonzero G-stable 
ideal / C k[X] is such that /"^ 7^ (0). Indeed, if F C X is a G-stable closed subset 
such that dominates X/^^ G, then vr(y) contains an open subset of Xj G and hence 
Y = 7r^^7r(y) contains an open subset of X; that is F = X. It follows that if y C X 
is a proper G-stable closed subset, then there exists z G (X/agG) \ vr(F). Let / G 
k[X/^g G] = k[Xf such that f{z) = 1 and f{n{Y)) = 0. Then / G J(F)^\{0}. □ 

The following two examples show that, in Theorem 13.101 both conditions 

(1) [k[Xf] = [k[Xf], and 

(2) Q{X) contains a nonempty open subset 

are necessary. 

Example 3.11. (1) Consider the action of GL„(k) on Mn(k) by left translations. 
Then [k[M„(k)]]^^"^"'^ = [k[M„(k)]GLn(k)] while we have that f](M„(k)) = 0. 

(2) Let G be a semisimple group, B C G a. Borel subgroup, and consider the 5-action 
on G by left translations. Then ^(G) = G while [k[G]^] = k C [k[G]]^. 



Remark 3.12. Observe that it follows from the proof of Theorem 13. 101 that for any 
/ G k[X]^ such that X/ C X^ax, then X/ C fi(X). 

With further assumptions, on either the algebraic group G or the affine variety X, 
there is a simplified characterization of the observability of the action G x X — > X. 

Proposition 3.13. Let G = L x U be a direct product of a reductive group with a 
unipotent group, and let X be a affine G-variety such that Q{X) contains a nonempty 
open subset. Then [k[X]'^] = [k[X]] ; that is, the action G x X X is observable. 

Proof. Let / C k[X] be a nonzero G-stable ideal. Then V(/) 7^ is a proper closed 
subset. Since there exists a open subset of closed orbits, then there exists a closed 
orbit Z such that Z fl V(/) = 0. Since L is reductive and that Z is L-stable, it 
follows that there exists / G k[X]^ such that / G /, /l^ = 1- Hence, ^ (0). 
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Since U normalizes L, it follows that / fl k[X]-^ 7^ (0) is an tZ-submodule, and hence 

jG ^ (jL)C/ _^ (0). □ 

Definition 3.14. Let fi : G x X X he a.n action of G on the affine variety X. We 
denote 

Eg{X) = |x G X{G) : there is a nonzero x-semiinvariant / G k[X]}. 

Corollary 3.15. Let fi : G x X ^ X be an action of G on the factorial affine variety 
X. Then the following are equivalent. 

(1) /i is observable. 

(2) (a) Q{X) contains a dense, open subset of closed G-orbits. 
(b) Eg{X) is a group. 

Proof. Assume that (1) holds. Then by Theorem 13.101 Q{X) contains a dense, open 
subset of closed orbits. Let / be a nonzero semiinvariant for x ^ X{G). If x ^ Eg{X) 
is not a unit in this monoid then the nonzero G-ideal /Ik:[X] C k[X] has no nonzero 
G-invariant, and thus /x is not observable. 

Conversely, assume that (2) holds, and let / be a nonzero G-ideal of k[X]. Let 
Guni be the closed normal subgroup of G generated by its unipotent elements. Since 
Guni is normal in G, any closed G-orbit of X consists of closed Guni-orbits. Thus, by 
Theorem 13.51 and Theorem 13.10^ J = J*^"'" is a nonzero ideal of k[X]'^"'". Now Gun; 
acts trivially on J, so that G acts on J through the torus S = G/Guni- Let / G J be 
a nonzero x-semiinvariant for this action. Since Eg{X) is a group there is a nonzero 
X~"'^-semiinvariant g G k[X]. Thus fgElr\ k[X]*^ = J"^ is the desired invariant. □ 

Corollary 3.16. Let ^ : G x X X be an action of G on the affine variety X and 
assume that G is solvable. Then the following are equivalent. 

(1) fi is observable. 

(2) Eg{X) is a group. 

Proof. The proof is similar to the proof of Corollary 13.151 taking into account the 
fact that, in this case, Guni is a unipotent group, which is always observable. □ 

The next result shows that observability is well behaved under induction of actions 
(see Definition 12.61) . 

Proposition 3.17. Let G be an affine algebraic group and H G G a closed subgroup, 
such that G/H is affine. Let H x X X be an action of H on the affine variety 
X. Then the action of H is observable if and only if the induced action of G on the 
affine variety G *h X is observable. 

Proof. First of all, observe that since G/H is affine, it follows that G*hX is an affine 
G-variety affine. Hence, k[G *h X] = (k[G] ® k[X])^, where if / ® ^ G k[G] ® k[X], 
then (a ■ (/ g)){b,x) = f{ha)g{a~^ ■ x), for all a G if, & G G, x G X. A simple 
calculation shows then that k[G *h = kigjklX]^ ^ k[X]^ . 

Assume now that the action H x X ^ X is observable and let Z C G *h X 
be a nonempty G-stable closed subset. If p : G *h X ^ G/H is the canonical 
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projection, then Z Ci p ^{IH) 7^ is a //-stable closed subset. It follows that there 

exists / e I{Z n p~\0))^ \ {0}. Then 1 O / G I{Zf \ {0}. 

Assume now that the action G G *h X is observable, and let y C X be a 
iJ-stable closed subset. Then GY G G *h X is a G-stable closed subset, and hence 
there exists 1 ® / G k O k[X]^ = k[G *h such that (1 (g) f){Z) = 0. It follows 
that /(F) = /(Znp-i(O)) = 0. □ 

As the final result of this section we show that, if the action is observable, there 
is a refinement in the construction of the open set Xq C X of Rosenlicht's Theorem 
13.41 In this case, there exists a G-stable principal affine open subset Xf C X with 
affine geometric quotient. 

Theorem 3.18. Let G an affine algebraic group acting on an affine variety X , such 
that the action is observable. Then there exists f G HsfX]*^ \ {0} such that k[X]j is a 
finitely generated k-algebra and Xf —>■ Xf/^^ G is a geometric quotient. 

Proof By Proposition EJl there exists / G k[X]^ \ {0} such that k[X]^ is a finitely 
generated algebra. Let Xq C X be an open subset as in Rosenlicht's Theorem. Then 
by Proposition 13.21 there exists g G k[X]'^ \ {0} such that Xg C Xq. It follows that 

Xfg C Xo is such that (k[X]/g) = (k[X]j)^ is finitely generated. 
Let h = fg and consider the commutative diagram 

Xh — ^ Xh/^y: G 
p 

Xh/G 

Then, 

k(X,/G) - k(X,)^ = [k[X,]f = [kfX],]"" = [k[X]^]. 
It follows that (fi is a birational morphism. 

Let q G k[X]j such that cp restricted to (p~^ [{Xh/ G)q) is an isomorphism 
over {Xh/^gG)g. Since 7i-'^{{Xh/ G) J = Xhq, it follows that p\^^^ : Xhg 
ip~^ [{Xh/ G)g) is the geometric quotient. □ 

4. The reductive case 

In what follows we apply the results of the previous section to the situation of a 
reductive group G acting on the affine variety X. We prove in Theorem 14 . 71 that there 
exists a unique, maximal, G-stable closed subset Xgoc C X such that the restricted 
action G x Xsoc Xsoc is observable. Furthermore, the categorical quotient Xsoc//G 
exists. See Theorem 14.61 
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Definition 4.1. Let G be an affine algebraic group acting on an algebraic variety 
X. We define the socle of the action to be 



Xsoc = U 0{X). 



'0(x)=0{x) 

Remark 4.2. Observe that (Xsoc)soc = ^soc- 

Lemma 4.3. Let G be a reductive group acting on an affine variety X , and vr : X — >■ 
X//G the categorical quotient. Then 

n{X) = TX-\{X//G) \ 7r(X \ X^,,)). 
In particular, VL{X) is a (possibly empty) G-stable open subset. 

Proof. It is clear that Vl{X) C 7r-i((X//G) \ 7r(X \ X^ax))- Since X \ X^^x is a G- 
stable closed subset, it follows that 7r(X\Xjnax) is closed in X//G. Let x G X\n(X); 
then 0{x) contains an unique closed orbit O' , with dimO' < dim (9. It follows that 
7r(x) = Ti{0') e 7r(X \ X^ax); that is x ^ 7r~'{iX//G) \ 7r(X \ X^ax))- □ 

Proposition 4.4 (Popov, [Hj). LetG be reductive group acting on an affine algebraic 
variety X, is such a way that f2(X) ^ 0. Then VL{X) is an open subset of X . 

Proof. See [HI Theorem 4] or Theorem 13.3.13]. □ 

Next we rephrase the contents of Proposition 14.41 in a obvious but useful charac- 
terization of condition (2) of Theorem 13. 101 namely that fi(X) contains a nonempty 
open subset. 

Lemma 4.5. Let G be a reductive group acting on an affine variety X , and vr : X — >■ 

X//G the categorical quotient. Then the following are equivalent. 

(1) T^\x\x^^^ '■ \ ^max ^ ^//G is dominant. 

(2) T!'\x\Xrm,,, ■ \ -^max ^//G is surjcctivc. 

(3) Xjnax does not contain any closed orbit. 

(4) Q{X) = 0. □ 

Theorem 4.6. Let G be a reductive group acting on an affine variety X , and Z G X 
a G-stable closed subset such that 7i\^ : Z ^ ^//G is dominant. Then the unique 
morphism cp : Z//G —>■ X//G, that fits into the following commutative diagram, 

Z^ ^X 



Z//G — X//G 

is a bijective, purely inseparable morphism. That is, (p is a bijective morphism, and 
if* : k[X]'^ k[Z]*^ induces a purely inseparable field extension [k[X]'^] C [k[Z]'^] . 

Moreover, Xsoc C Z , Xgoc is irreducible, and f2(Xsoc) 7^ 0- 

In particular, z/chark = 0, then tt\^ : Z X//G is the categorical quotient and 
ip : Xsoc//G — s> X//G is an isomorphism. 
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Proof. Since Z C X is closed, it follows that vr(Z) C X//G is closed, and hence 
7i{Z) = X//G. In particular, if O C X is a closed orbit, it follows that vr(0) G vr(Z) = 
X, and hence there exists an orbit O' & Z such that '/r((9') = vr((9). Therefore, since 
71 separates closed orbits, it follows that O' = O C Z and hence Xgoc C Z. 

Since ip o p is dominant, it follows that ip is dominant. Since vr and p separate 
closed orbits, then cp is injective. Thus, we have the following commutative diagram: 




Since G is reductive, k[Z]*^ is finitely generated and there exists n > such that 
for all / G Ik:[Z]'^, there exists g G k[X]'^, with (f{g) = (see for example [ini 
§9.2]). Hence tt\^ : Z ^ ^//G is a purely inseparable morphism. It follows that if 
chark = 0, then X//G ^ Z//G. 

Since vr|^_^^^ : X^oc X//G is surjective, it follows that ip : Xsoc//G X//G is a 
purely inseparable morphism. 

Let Z C Xsoc an irreducible component such that 7r|^ is dominant. Then X^oc C Z, 
and hence Xgoc is irreducible. 

Consider Z = Xsoc \ (Xsoc)max- If Z dominates Xsoc//G, it follows that Xgoc = 
(Xsoc)soc C Z, that is a contradiction. Hence, by Lemma IT5l it follows that fi(Xsoc) 7^ 
0. □ 

Theorem 4.7. Let G be reductive group acting on an affine algebraic variety X. 
Then the action is observable if and only if VL{X) 7^ 0. In particular, X^oc is the 
maximum G-stable closed subset Z C X such that the restricted action G x Z Z 
is observable. 

Proof. If the action is observable, it follows from Theorem 13.101 that Q{X) 7^ 0. 
Assume now that Q{X) 7^ and let Z C X be a G-stable closed subset. If f2(X) C Z 
it follows that Z = X; hence Q{X) \ Z 7^ 0. Recall that the categorical quotient 
71 : X ^ X//G = Spec(k[X]*^) separates closed orbits. It follows that Q{X) \ 
7r~^(7r(Z)) 7^ 0, since the closed orbits belonging to Z and 7r~^(7r(Z)) are the same. 
Let O C Q{X)\Z be a closed orbit. Then 7r~^(7r(0)) = O, again because vr separates 
closed orbits. Since tt also separates saturated closed subsets, it follows that there 
exists / G k[X]^ such that / G l{7i~^{7i{Z))) C I{Z) and f{0) = 1; in particular, 

fel{zr\{0}. 

It follows by the very definition of Xgoc (e.g. from Proposition that f2(Xsoc) 7^ 0- 
Let now Z be a G-stable irreducible closed subset such that the restricted action is 
observable; then Q{Z) is a nonempty open subset of Z, consisting of closed orbits in 
Z, and hence in X. It follows that Z = Q{Z) C Xgoc- If Y is any G-stable closed 
subset, it follows by Lemma [3.11 that the restriction of the action to any irreducible 
component Z is observable, and hence Y C Xgoc- D 
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Proposition 4.8. Let G be a reductive group acting on an affine variety X . Then 
X(Xsoc) is the maximum G-stable ideal such that = (0). 

Proof. Let / = J2jg=(o) J be the sum of all G-stable ideals such that = (0), and 
consider the canonical G-morphism ip : 0jg=(o) J I- Since ip is surjective, it 

follows that for every f E I'^ there exist n > and h G (©jg=(o) J)'~^ such that 

^(h) = fP\ where chark = p. But (0jg=(o) J)^ = (0). Hence, = (0). 

Let O C X he a closed orbit and assume that Cfl V(/) = 0. Since k[X]'^ separates 
G-stable closed subsets, if follows that there exists / G k[X]'-^ such that /l^ = 1 and 
f\v(i) ~ hence /"^ 7^ (0) and we get a contradiction. Therefore, Xg^c C V(/). 

Observe that if / G r{I)^ is such that /" G /, it follows that for any a G G, 
then a ■ (/") = /" G /, and hence / = 0. Thus, r{I)'^ = (0) and by maximality 
then I = r{I). By Theorem 14.71 if we prove that the action G x V(/) is 
observable, then Xgoc = V(/). But k[V(/)] = k[X]/J, and hence G-stable ideals of 
k[V(/)] are of the form J/ 1, were J C k[X] is a G-stable ideal containing I. Then 
if J/J 7^ (0) it follows that I ^ J and hence, by maximality of /, 7^ (0). Thus, 
(J/I)G ^ (0), since k[Xf injects in k[X]//. □ 
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